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Enrolling in college and completing a degree or certificate is one of the primary pathways to economic success. In 2017, 
median weekly earnings of full-time workers with an associate degree was 17 percent higher than full-time workers with a high 
school diploma only.' Despite the economic advantages of educational attainment, only 32 percent of students enrolled in 
two-year institutions complete their programs within three years. The first year of college is critical, as 38 percent of first-time, 
full-time students in 2-year institutions fail to return to the same institution for their second year. As a result, several programs 
have been designed to support student retention and degree completion among college students. One example is the City 
University of New York's (CUNY’s) Accelerated Study in Associate Programs (ASAP). 


ASAP is a three-year program that is designed to remove barriers to college success and completion for students seeking associate 
degrees. ASAP offers students financial supports (tuition waivers, free use of textbooks, free public transportation), academic 
supports (tutoring, early registration), and personal supports (advisors, seminars, blocked or linked courses). ASAP students are 
required to enroll full time and are encouraged to take any required developmental education courses in the first semester. 


This What Works Clearinghouse (WWC) report, part of the WWC's Supporting Postsecondary Success topic area, explores the 
effects of ASAP on enrollment, credit accumulation, and graduation rates. The WWC identified 11 studies of ASAP, two of which 
met WWC standards.’ The evidence presented in this report includes studies of the impacts of ASAP on community college 
students, including students who were White, Black, Hispanic, and Asian or Pacific Islander. 


What Happens When Students Participate in ASAP?* 


The evidence indicates that implementing ASAP: 

- will likely increase student graduation rates 

¢ will likely increase student enrollment rates 

* will likely increase student credit accumulation and persistence rates 

Findings on ASAP from two studies that meet WWC standards are shown in Table 1. For each outcome reviewed by the WWC, 
an effectiveness rating, the improvement index, and the number of studies and students that contributed to the findings 

is presented. The improvement index is a measure of the intervention’s effect on an outcome. It can be interpreted as the 


expected change in percentile rank for an average comparison group student if that student had received the intervention. 
All findings are based on two studies with 2,397 students. See Box 1 for a description of WWC effectiveness ratings. 


Table 1. Summary of findings on ASAP from studies that meet WWC Standards 


Evidence meeting 
Study findings WWC standards (version 3.0) 

Improvement index Number of Number of 
Outcome Effectiveness rating (percentile points) studies students 
Attainment Positive effects +17 2 2,397 
Access and enrollment Positive effects +13 2 2397 
tion and persistence Positive effects +11 2 2,397 

Table Note: The improvement index is a measure of the effect of the intervention. The improvement index can be interpreted as the expected change in percentile rank for an average comparison 


oup student if that student had received the intervention. For example, an improvement index of +17 means that the expected percentile rank of the average comparison group student would 


Credit accumula 


crease by 17 points if they received ASAP. The improvement index values are generated by averaging findings from the outcome analyses that meet WWC standards, as reported by Scrivener et 


g 

al. (2015) and Sommo et al. (2018). Attainment outcomes include completion of a degree, certificate, or program. Access and enrollment outcomes include enrollment in college the first semester 
after graduating high school and enrollment in a four-year college. Credit accumulation and persistence outcomes include the number of college-level credits earned, number of sessions enrolled, 
; ; 
a 


d enrollment in the fourth semester. The effects of ASAP are not known for other outcomes within the Supporting Postsecondary Success topic area, including college attendance, academic 
chievement, and labor market outcomes. 
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BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria that the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 3.0) and the Supporting Postsecondary Success topic area protocol (version 3.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and how many 
studies tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported results and about 
what will happen if the same intervention is implemented again. The following key provides a link between effectiveness ratings and 
the statements used in this report: 


Effectiveness Rating Rating Interpretation Description of the Evidence 


Positive (or Negative) Effects The intervention is likely tochange Strong evidence of a positive effect, with no 
an outcome overriding contrary evidence 


Potentially Positive (or Negative) Effects The intervention may change an Evidence of a positive effect with no overriding 
outcome contrary evidence 


No Discernible Effects The intervention may result in little | No affirmative evidence of effects 
to no change in an outcome 


Mixed Effects The intervention has inconsistent Evidence includes studies in at least two of 
effects on an outcome these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


How is ASAP Implemented? 


The following section provides details of how ASAP was implemented. This information can help educators identify the 
requirements for implementing ASAP, and determine whether those implementation requirements would be feasible at their 
institutions. Information on ASAP presented in this section comes from the two studies that meet WWC evidence standards 
(Scrivener et al., 2015; Sommo et al., 2018), from CUNY’s ASAP website, and from correspondence with the developer. 


* Goal: ASAP aims to help students earn associate degrees within three years by providing a range of financial, academic, and 
personal supports to remove barriers to college success and completion. 


* Target Population: ASAP is broadly targeted to low-income, full-time students in need of developmental coursework. 
Students admitted to CUNY colleges that offer ASAP can participate in the program provided they: (1) complete the Free 
Application for Federal Student Aid (FAFSA), the New York State Tuition Assistance Program (TAP), and all admissions 
requirements; (2) need to take no more than two developmental courses; (3) enroll full-time; (4) complete a mandatory 
summer institute before enrolling; and (5) sign a contract agreeing to the ASAP terms of participation upon completion of 
the institute. Continuing or transfer students are eligible if they have GPAs of at least 2.0 and have accumulated no more 
than 15 college credits. To remain eligible, students must renew their FAFSA and TAP annually, maintain full-time status 
and be in good academic standing, enroll in all developmental courses within the first year, and meet regularly with their 
advisor, career specialist, and tutors as well as attend required program enrichment activities. 


« Method of Delivery: ASAP students receive mentoring and guidance from an experienced advisor. The advisor provides 
comprehensive academic, social, and interpersonal supports. In addition to an advisor, ASAP facilitates relationship 
building among students through advising groups, a one or two-day required summer institute, and blocked courses in 
which students take classes with other ASAP students in a cohort model. Students can also be connected to tutors, career 
development specialists, and other campus-based supports, as needed. 


« Frequency and Duration of Service: Students meet with ASAP advisors at least twice Comparison Group: In the 
per month in their first semester. After the first semester, students meet with their two studies that contributed 
advisors on a regular basis depending on their need level, from once every two months to the findings, comparison 
for students in the low-need group to twice per month for students in the high-need group students received 
group. Students participate in ASAP for three years. the usual college advising 

services provided by their 


* Intervention Components: The ASAP model includes three key components, which 


are noted in Table 2. institutions. 
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Table 2. Components of ASAP 


Key component ASAP 


Financial Eligible students receive supports to address financial barriers to degree completion. These supports include tuition waivers to cover any 

supports gap in need between financial aid and the cost of tuition and fees; textbook assistance to offset the cost of books (on average students 
receive $500 annually); and support with summer and winter tuition to facilitate continuous enrollment. CUNY students receive a semester 
MetroCard to ensure they have transportation to campus. 


Academic ASAPS academic supports are based on a structured pathways model. Students begin by enrolling full-time in a major that can be 

supports completed in no more than three years (most majors except nursing and allied health meet this criterion). They work with an academic 
advisor to develop a schedule that allows them to complete courses in a prescribed sequence with a cohort of other students in the 
program. ASAP advisors encourage students to take courses in winter and summer terms (continuous enrollment). ASAP staff coordinate 
with campus colleagues to provide ASAP students opportunities for early registration. 


Students must complete assigned developmental coursework in their first year. Students needing two developmental courses must take at 
least one of those courses in the summer prior to their first year. Advisors reserve spaces in developmental courses for ASAP students. 


Personal Students meet with an assigned advisor regularly at intervals determined based on student needs. Advisors maintain average caseloads of 
supports 150 students so they can provide personalized, consistent, and comprehensive support to each student. 


In addition to providing an advisor, ASAP facilitates relationship building among students through advising groups, a required one or two- 
day summer institute, and blocked courses in which students take classes with other ASAPstudents. 


Students are also connected to tutors, career development specialists, and other campus-based supports, as needed. ASAP provides 
additional services to support transfer, career readiness skill-building, and leadership development opportunities through participation in 
ASAP Student Leader and Peer Mentor programs. 


What Does ASAP Cost? 


The cost of ASAP is reported in one study reviewed. Scrivener et al. (2015) calculated the direct cost per ASAP student at 
$14,029 over a three-year period ($4,676 per year). The cost breakdown below is not designed to be exhaustive; rather, it is 
designed to provide educators an overview of the kinds of resources needed to implement ASAP. 


ASAP 


Personnel Costs: The three-year cost includes $6,238 per student on administration and staffing (e.g., management, 
research and evaluation), $1,558 on course enrollment (e.g., blocked or linked courses, ASAP seminar), and $2,927 on student 
services (e.g., advising, tutoring, career and employment services). 


Facilities Costs: Beyond the facilities costs normally associated with college attendance, ASAP hosts a one or two-day 
summer institute for students and hosts the ASAP Student Leader and Peer Mentor programs. 


Equipment and Materials Costs: ASAP does not incur any equipment and materials costs beyond those normally 
associated with college attendance. 


Costs Paid by Students or Parents: There are no direct program costs billed to students or parents; however, students 
must pay for costs normally associated with college attendance that are not covered by the ASAP program. 


In-Kind Supports: The three-year cost for in-kind financial supports including MetroCards, textbooks, and tuition waivers 
totals $3,305. 


Sources of Funding: |n addition to city and state support, ASAP has received funding from a range of foundations: the 
Robin Hood Foundation, the Stella and Charles Guttman Foundation, the Leona M. and Harry B. Helmsley Charitable Trust, 
the Jewish Foundation for the Education of Women, and the Sidney and Laura Gilbert Scholarship Fund. 


For More Information: 


About ASAP 


The City University of New York, Accelerated Study in Associate Programs 
16 Court Street, 32nd Floor, Brooklyn, NY 11241-0102 
Web: http://www'1.cuny.edu/sites/asap/ 
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For More Information (continued): 
About the cost of the intervention 


Levin, H.M. & Garcia, E. (2018). Accelerating community college graduation rates: A benefit—cost analysis. The Journal of Higher Education, 89(1), 1-27, 
DOI: 10.1080/00221546.2017.1313087. 


Scrivener, S., Weiss, M.J., Ratledge, A., Rudd, T., Sommo, C., & Fresques, H. (2015). Doubling graduation rates: Three-year effects of CUNY’s Accelerated 
Study in Associate Programs (ASAP) for developmental education students. New York: MDRC. 


Research Summary 


The WWC identified 11 studies that investigated the effectiveness of ASAP (Figure 1): 

+ 2studies meet WWC group design standards without reservations 

+ Ostudies meet WWC group design standards with reservations 

+ 2studies do not meet WWC group design standards 

+ 7 studies are ineligible for review 

The WWC reviews findings on an intervention’s effects on eligible outcome domains from studies that meet WWC group 
design standards, either with or without reservations. Based on this review, the WWC generates an effectiveness rating, which 


summarizes how the intervention impacts, or changes, a particular outcome domain. Findings from studies that either do not 
meet WWC standards or are ineligible for review do not contribute to the effectiveness ratings. 


The two studies of ASAP that meet WWC group design standards reported findings on credit accumulation and persistence, 
access and enrollment, and attainment. The studies did not report findings on the following three domains in the Supporting 
Postsecondary Success topic area: college attendance, academic achievement, and labor market outcomes. Citations for all 11 
studies reviewed for this report are listed in the References section, which begins on page 10. 


Figure 1. Effectiveness ratings for ASAP 


studies meet WWC studies meet WWC studies do not studies are 

standards without standards with meet WWC ineligible for 

reservations reservations standards review 
Contributes to Effectiveness Ratings Do Not Contribute to Effectiveness Ratings 


ee 


ASAP has positive effects on attainment 


The WWC determined that two studies that meet WWC group design standards without reservations showed evidence of a positive 
and statistically significant effect of ASAP on attainment (Scrivener et al., 2015; Sommo et al., 2018). 


ASAP has positive effects on college access and enrollment 


The WWC determined that two studies that meet WWC group design standards without reservations showed evidence of a positive and 
statistically significant effect of ASAP on college access and enrollment (Scrivener et al., 2015; Sommo et al., 2018). 


ASAP has positive effects on credit accumulation and persistence 


The WWC determined that two studies that meet WWC group design standards without reservations showed evidence of a positive and 
statistically significant effect of ASAP on credit accumulation and persistence (Scrivener et al., 2015; Sommo et al., 2018). 
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Main Findings 

Table 3 shows the findings from two ASAP studies that meet WWC evidence standards. The table includes WWC calculations of 
the mean difference, effect size, and improvement index. Based on those studies, the effectiveness rating for the attainment, 
access and enrollment, and credit accumulation and persistence outcome domains is positive effects, indicating that there is 
evidence of ASAP'’s positive effects on each domain with no overriding contrary evidence. These findings are based on two 
studies with 2,397 students. 


Table 3. Findings from studies of ASAP by outcome domain 


Mean 
(standard deviation) WWC calculations 

Study Sample Intervention Comparison Mean Effect +Improvement 
Measure (study) sample size group group _ difference size index p-value 
Famed any degree within six years Full sample 896 51.9 40.4 115 0.28 +11 <.01 
(%) students 
(Scrivener et al., 2015)? 
Earned a degree or certificate within Full sample 1,501 19.0 79 11.1 (0.61 +23 <.01 
2 years (%) students 
(Sommo et al., 2018)° 
Domain average for attainment across all studies 0.45 +17 
Enrollment in first semester (%) Full sample 896 96.7 94.2 25 0.36 +14 076 
(Scrivener et al., 2015)? students 
Enrollment in a four-year college, Full sample 86 25.1 173 78 = 0.29 +11 004 
sixth semester (%) students 
(Scrivener et al., 2015)? 
Enrollment in first semester (%) Full sample 1,501 95.1 91.4 3.6 0.36 +14 <.01 
(Sommo et al., 2018)? students 
Domain average for access and enrollment across all studies 0.34 +13 
Number of credits earned over three Full sample 86 42.9 35.2 77 ~~ 0.28 +11 <.0001 
years students (27.7) (27.8) 
(Scrivener et al., 2015)? 
Number of sessions enrolled over Full sample 86 6.6 54 12 0.39 +15 <.0001 
three years students (3.0) (3.1) 
(Scrivener et al., 2015)? 
Enrollment in fourth semester (9) Full sample 1,501 60.2 50.5 97 0.24 +9 <.01 
(Sommo et al., 2018)? students 
Domain average for credit accumulation and persistence across all studies 0.29 +11 


Table Notes: For mean difference, effect size, and improvement index values reported in the table, a positive number favors the intervention group and a negative number favors the comparison 
group. The effect size is a standardized measure of the effect of an intervention on outcomes, representing the average change expected for all individuals who are given the intervention (measured 
in standard deviations of the outcome measure). The WWC-computed average effect size is a simple average rounded to two decimal places; the average improvement index is calculated from the 
average effect size. The improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention. The 
Statistical significance of the domain average was determined by the WWC. Some statistics may not sum as expected due to rounding. 


* For Scrivener et al. (2015), a correction for multiple comparisons was needed but did not affect whether any of the contrasts were found to be statistically significant. No corrections were needed 
for clustering or to adjust for baseline differences. The p-values presented here were reported in the original study. This study is characterized as having a statistically significant positive effect in all 
three domains because the effect for at least one measure within each domain is positive and statistically significant, and no effects are negative and statistically significant, accounting for multiple 
omparisons. 


For Sommo et al. (2018), no corrections were needed for multiple comparisons, clustering, or to adjust for baseline differences. The p-values presented here were reported in the original study. This 
udy is characterized as having a statistically significant positive effect in all three domains because the effect for at least one measure within each domain is positive and statistically significant, and 
0 effects are negative and statistically significant, accounting for multiple comparisons. 


C 
b 
S 


For more information, please refer to the WWC Procedures and Standards Handbook (version 3.0), page 26. 
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In What Context Was ASAP Studied? 


The following section provides information on the setting and participants involved in the two studies of ASAP that meet WWC 
evidence standards. This information can help educators understand the context in which the studies of ASAP were conducted 
so that they can better determine whether the program might be suitable for their setting. 


WHERE THE STUDY WAS CONDUCTED 


@ A 2 studies, 2,397 students in © community colleges in New York City and Ohio 


RACE ETHNICITY 


Minority White Hispanic Non-Hispanic 


GENDER CHILDREN 


Female Male Have Children Do Not 


GRADES: Pk DMC @ 
Postsecondary (PS) 


Details of Each Study that Met WWC Standards 


This section presents details for each study of ASAP that meets WWC standards. These details include the full study reference, 
findings description, findings summary, and description of study characteristics. A summary of findings for each outcome 
domain examined is presented in the first table for each study. The second table for each study provides a description of 

the study characteristics. These study-level details include contextual information on the study setting, methods, sample, 
intervention group, comparison group, outcomes, and implementation details. For additional information, the reader should 
refer to the original studies. 


Research details for Scrivener et al. (2015) 


Scrivener, S., Weiss, MJ., Ratledge, A., Rudd, T., Sommo, C., & Fresques, H. (2015). Doubling graduation rates: Three-year effects of 
CUNY’s Accelerated Study in Associate Programs (ASAP) for developmental education students. New York: MDRC. 


Findings from Scrivener et al. (2015) show evidence of a positive effect of ASAP on attainment, access and enrollment, and 
credit accumulation and persistence outcomes (Table 4). The improvement index is a measure of the effect of the intervention. 
For example, an improvement index of +11 means that the percentile rank of the average comparison group student would 
improve by 11 points on attainment outcomes if they received ASAP. These findings are based on one outcome in the 
attainment domain, two outcomes in the access and enrollment domain, and two outcomes in the credit accumulation and 
persistence domain. All findings are based on outcome analyses that include 896 students. 
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Table 4. Summary of findings from Scrivener et al. (2015) 


Meets WWC group design standards without reservations 


Study findings 
Average Improvement Statistically 
Outcome domain Sample size effect size index significant 
Attainment 896 community college students 0.28 +11 Yes 
Access and enrollment 896 community college students 0.32 +13 Yes 
Credit accumulation and persistence 896 community college students 0.34 +13 Yes 


Table 5. Description of study characteristics for Scrivener et al. (2015) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 
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Meets WWC Group Design Standards Without Reservations. [his is a randomized controlled trial (RCT) with low 


attrition. 


The study was conducted at three CUNY community colleges: Bo 


Community College (KCC), and LaGuardia Community College (LGC). 


The study uses a randomized contro 
study if 
Pell grant; (2) they needed one or two developmental courses in 
fewer college credits; (4) they were 


rough of Manhattan Community College (BMCC), Kingsborough 


ed trial (RCT) design. Students in the three CUNY community colleges were eligible for the 
they met the following criteria: (1) they had a family income below 200 percent of the poverty 


evel or were eligible for a 


math, reading, or writing; (3) they had previously earned 12 or 


ew York City residents; (5) they were willing and able to attend college full time; and (6) they 


were in an ASAP-eligible major. Prior to the spring 2010 semester and fall 2010 semester, eligible students were asked to attend an 
intake session on campus where they completed an informed consent form and a Baseline Information Form. Students were then 
randomly assigned to either receive an offer of participating in ASAP or to serve in a business-as-usual comparison group. Random 


assignment took place at BMCC and KCC for the spring 2010 sem 


ester and the fall 2010 semester; in LGCC, random assignment only 


took place for the fall 2010 semester. Altogether, 451 students were randomly assigned to the intervention group and 445 students 
were randomly assigned to the comparison group across the two semesters. 


Among the 896 students in the evaluation sample, 38 percent were male and 62 percent were female. At baseline, the average age 
of students was 21.5 years, 44 percent of students were Hispanic, 10 percent were White, 34 percent were Black, 8 percent were 
Asian or Pacific Islander, and 5 percent fell into the “Other” category. Moreover, 60 percent of students were incoming freshmen, 88 
percent received a Pell Grant, and at least 86 percent of students had a developmental need in one or two classes (developmental 


need was unknown for 11 percent of students). 


The ASAP intervention is a three-year multi-component program that provides wraparound supports to incoming community 
college students, including: tutoring; advising; career and employment services; blocked or linked courses; seminars; early 


enrollment; tuition waivers; free use of textbooks; and free public transportation. Students are required to attend college full time 
and are strongly encouraged to take developmental courses early. ASAP does not aim to alter classroom instruction. ASAP has three 


primary components: 


1. Academic Supports: Students enroll in blocked or linked courses in their first year (i.e., multiple courses offered back-to-back or 
that have connected content and keep ASAPstudents together in the same courses). Students can also register for courses early 


and enroll in an ASAP seminar during the first three semeste 


rs which covers topics such as student skills and goal setting. 


2. Financial Supports: Students receive a tuition waiver that covers the gap between financial aid and college tuition and fees (an 
average of $719 per semester among the small percentage of students who received the waiver). Students also receive free 
public transportation for subway and bus travel (Metro Cards, which cost an average of $112 per month by the end of the third 


year of the study) and free use of textbooks (worth approximately $270 per semester). 


3. Personal Supports: In their first year, students received advisement from an ASAP advisor at least twice per month (average: 


38 meetings per student during that time), support from an 
once per semester (average: nine meetings during that time 


ASAP career and employment services staff member at least 
_ and one hour per week of tutoring from ASAP staff for each 


developmental course taken (tutoring was also required when students’ GPA fell below 2.0) (average: 24 sessions during that 


time). 


In addition, ASAP provided voluntary services to students, includi 
Transfer Scholarship to complete degrees at CUNY four-year colle 
develop leadership and public speaking skills. Neither of these of 


ng access to a social work intern, a $500 per semester ASAP 
ges, and participation in an ASAP Student Leadership Program to 
ferings was widely used by students. 


Table 5. Description of study characteristics for Scrivener et al. (2015) 


Comparison Unlike ASAP, students in the comparison condition were not required to enroll full time. Students in the comparison group had 


group access to the usual college services: On average in the 
advisor, two meetings with career services, and seven 


ir first year, students in the comparison group had six meetings with an 
tutoring sessions. Blocked or linked courses were available at two of the three 


colleges during the first semester, but enrollment in these courses is unknown. Some students also took a student success course 
or a freshman seminar during their first year. Students in the comparison group had no access to additional financial supports (i.e., 
tuition waivers, free MetroCards, or free use of textbooks). 


Outcomes and The study includes one eligible outcome in the attainment domain (earned a degree), two eligible outcomes in the access and 

measurement enrollment domain (enrollment in first semester, enrollment in a four-year college), and two eligible outcomes in the credit 
accumulation and persistence domain (total number of college-level credits earned, number of sessions enrolled). All outcome 
measures are standard educational measures, pulled from student records, with assumed reliability. Data were drawn from CUNY’s 


Institutional Research Database and the National Stud 


The measure of college-level credits earned is a cumu 


ent Clearinghouse. 
ative measure. This measure excludes courses that were passed more than 


once. Degree receipt is also a cumulative measure; any students who earned a degree in a given semester are counted as having 


earned a degree in all subsequent semesters. 


Subgroup analyses were presented by gender and hig 
WWC website (https://whatworks.ed.gov) and 


h school diploma receipt. These supplemental findings are reported on the 
do not factor into the intervention’s rating of effectiveness. 


A number of outcomes were not eligible for review under the Supporting Postsecondary Success protocol: (1) completion of 
developmental education requirements; (2) developmental credits earned; (3) total credits earned (since this outcome includes 


developmental credits); (4) completed developmenta 
attempted; and (7) cost effectiveness. 


education requirements; (5) enrolled at any CUNY college; (6) total credits 


Additional ASAP was initially funded in 2007 with a three-year, $20 million grant from the New York City Center for Economic Opportunity. 
implementation —_ASAPis jointly administered by the CUNY Office of Academic Affairs and the participating community colleges 


details 


Research details for Sommo et al. (2018) 


Sommo, C., Cullinan, D., & Manno, M. (2018). Doubling graduation rates in a new state: Two-year findings from the ASAP Ohio 
demonstration. MDRC Policy Brief, December. Retrieved from https://www.mdrc.org/sites/default/files/ 


ASAP brief 2018 Final.pdf 


Findings from Sommo et al. (2018) show evidence of a positive effect of ASAP on attainment, access and enrollment, and credit 
accumulation and persistence (Table 6). The improvement index is a measure of the effect of the intervention. For example, 


an improvement index of +23 means that the percentile rank of 


the average comparison group student would improve by 23 


points on attainment outcomes if they received ASAP. These findings are based on one outcome in the attainment domain, 
one outcome in the access and enrollment domain, and one outcome in the credit accumulation and persistence domain. Alll 
findings are based on outcomes analyses that include 1,501 students. 


Table 6. Summary of findings from Sommo et al. (2018) 


Meets WWC group design standards without reservations 


Study findings 
Average Improvement Statistically 
Outcome domain Sample size effect size index significant 
Attainment 1,501 community college students 0.61 +23 Yes 
Access and enrollment 1,501 community college students 0.36 +14 Yes 
Credit accumulation and persistence 1,501 community college students 0.24 +9 Yes 
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Table 7. Description of study characteristics for Sommo et al. (2018) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Comparison 
group 


Outcomes and 
measurement 


Additional 
implementation 
details 


Meets WWC Group Design Standards Without Reservations. This is a randomized controlled tria 
attrition. 


(RCT) with low 


The study was conducted at three Ohio community colleges: Cincinnati State Technical and Community College, Lorain County 
Community College, and Cuyahoga Community College. 


Students at three Ohio community colleges were eligible to participate in the ASAP evaluation if the following criteria were met: (1) 
they planned to enroll in spring 2015, fall 2015, or spring 2016; (2) they were Pell Grant eligible (i.¢., low income); (3) they were 
degree-ready or in need of developmental education; (4) they were planning to enroll full-time; and (5) they were in a major that 
can be completed in three years. Prior to start of the semester, eligible students were contacted by college advisers and staff who 
explained the program and determined their interest. If interested, students completed an informed consent form and a baseline 
questionnaire. Students were then were then randomly assigned to either receive an offer of participating in ASAP or to serve ina 
business-as-usual comparison group. Altogether, 806 students were randomly assigned to the intervention group and 695 students 
were randomly assigned to the comparison group across the two semesters. 


Among the 1,505 students in the study sample at baseline, 36 percent were male and 64 percent were female. At baseline, 46 
percent of students were White, 35 percent were Black, 10 percent were Hispanic, and 10 percent fell into the “Other” category. 
Moreover, 60 percent of students were employed at baseline, 9 percent were married, 27 percent had children, 47 percent were 
nontraditional students, and 34 percent were the first person in their family to attend college. 


The ASAP intervention is a three-year multi-component program that provides wraparound supports to incoming community 
college students, including: tutoring; advising; career and employment services; blocked or linked courses; seminars; early 
enrollment; tuition waivers; and free use of textbooks. This intervention model in Ohio differs from the CUNY ASAP model primarily 
in two ways: (1) there is no requirement or encouragement of enrollment in a winter session as none of the participating Ohio 
community colleges offer a winter session, and (2) program oversight management is decentralized in Ohio at each of the three 
campuses whereas CUNY ASAP management is central at the CUNY Office of Academic Affairs. 


Unlike ASAP, students in the comparison condition were not required to enroll full time and could not participate in ASAP programs. 
Students in the comparison group had access to the usual college services. 


J 


The study includes one eligible outcome in the attainment domain (earned a degree or certificate in the fourth semester), one 
eligible outcome in the access and enrollment domain (enrollment in the first semester), and one eligible outcome in the credit 
accumulation and persistence domain (enrollment in the fourth semester). All outcome measures are standard educational 
measures, pulled from student records, with assumed reliability. Data were drawn from transcripts from the three participating 
community colleges. 


Supplemental analyses presented enrollment rates for the second and third semester, and by full-time vs. part-time status. 
Graduation outcomes in semesters 2-3 were also presented. These supplemental findings are reported on the WWC website 
(https://whatworks.ed.gov) and do not factor into the intervention’s rating of effectiveness. 


The study also included measures of credit accumulation in each semester. Since these measures of credit accumulation included 
developmental credits in addition to college-level credits, they are out of the scope of the Supporting Postsecondary Success 
protocol, version 3.0. 


ASAPin Ohio was funded through a consortium set up by MDRC. The primary funder was the Great Lakes Higher Education 
Guaranty Corporation with additional funding provided from the Bill & Melinda Gates Foundation, the Ford Foundation, the Greater 
Cincinnati Foundation, Haile U.S. Bank Foundation, KnowledgeWorks, the Kresge Foundation, and the Lumina Foundation. Over 
time, the participating community colleges will take on an increasing share of the costs. CUNY and MDRC provided start-up 
technical assistance and the Ohio Department of Higher Education served as coordinator of the Ohio ASAP Network to facilitate 
communication and coordination across participating community colleges. 
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Endnotes 


' Bureau of Labor Statistics. (2018). Measuring the value of education. Washington, DC: Author. Retrieved from 
https://www.bls.gov/careeroutlook/2018/data-on-display/education-pays.htm 

* McFarland, J., Hussar, B., Zhang, J., Wang, X., Wang, K., Hein, S., Diliberti, M., Forrest Cataldi, E., Bullock Mann, F., and Barmer, A. (2019). The Condition of 
Education 2019 (NCES 2019-144). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved from 
https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2019144 

>The descriptive information for this intervention comes from Scrivener et al. (2015) and Sommo et al. (2018), as well as from ASAP's website. The 
What Works Clearinghouse (WWC) requests developers review the intervention description sections for accuracy from their perspective. The WWC 
provided CUNY with the intervention description in April 2019 and incorporated feedback from CUNY's response. Further verification of the accuracy 
of the descriptive information for this intervention is beyond the scope of this review. 

‘The literature search reflects documents publicly available by March 2019. Reviews of the studies in this report used the standards from the 


WWC Procedures and Standards Handbook (version 3.0) and the Supporting Postsecondary Success review protocol (version 3.0). The 
ed in this report is based on available research. Findings and conclusions could change as new research becomes available. 


evidence presen 
° This study used a comparison group from the previous school year, and thus has a time confound, for the 2007, 2009, and 2010 cohorts. Starting 
with the fall 2011 cohort, the authors matched ASAP students with comparison students from the same school year, so later cohorts in this study do 
not have a fundamental confound. The WWC cannot establish baseline equivalence on these later cohorts because the Supporting Postsecondary 
Success Protocol (version 3.0) requires baseline equivalence to be established on a measure of socioeconomic status and a continuous measure 
of academic achievement for outcomes without a natural pretest. Since the only academic achievement pretest data available were the CUNY 
placement tests in reading, writing, and mathematics, and since some students were exempt from taking this test, a continuously-scaled pretest for 
the analytic sample was not available. Data provided by the authors that imputed test scores for students who placed out of these subjects suggests 


that the treatment and comparison groups were well-balanced at baseline. 
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